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We  Rely  on  Software  for  Safe  Aircraft  Operation 


Quantas 

Landing 

Written  by  htbv 
From:  soya  wan 


from  Singapore  to 


Even  with  the  autopilot  off,  flight  control  computers  still  '  '  command  control 
surfaces  to  protect  the  aircraft  from  unsafe  conditions  such  as  a  stall/1  the 
investigators  said. 

The  unit  continued  to  send  false  stall  and  speed  warnings  to  the  aircraft's 
primary  computer  and  about  2  minutes  after  the  initial  fault  '  '  generated 
very  high,  random  and  incorrect  values  for  the  aircraft's  angle  of  attack." 

mayday  call  when  it  suddenly  changed  altitude  during  a  flight 
Qantas  said. 


Embedded  software  systems 
introduce  a  new  class  of 
problems  not  addressed  by 
traditional  system  modeling  & 


Liu. 


analysis 

, .  -  - 1- -■ 


lunge 


wide 
imays 

causing  the 


Autopilot  Off 

A  "  "  preliminary  analysis"  of  the  Qantas  plunge  showed  the  error  occurred 
in  one  of  the  jet's  three  air  data  inertial  reference  units,  which  caused  the 
autopilot  to  disconnect,  the  ATSB  said  in  a  statement  on  its  Web  site. 

The  crew  flew  the  aircraft  manually  to  the  end  of  the  flight,  except  for  a 
period  of  a  few  seconds,  the  bureau  said. 


jet  to  nosedive. 


vas  cruising  at  37,000  feet  (11,277  meters) 
computer  fed  incorrect  information  to  the  flight  control  system,  the 
Australian  Transport  Safety  Bureau  said  yesterday.  The  aircraft  dropped 
650  feet  within  seconds,  slamming  passengers  and  crew  into  the  cabin 
■~-~iilini~|,  hrfnr--1  p i I r~-g~iin-d  rnni~Tli 


Even  with  the  autopilot  off,  flight  control  computers  still  "  "  command  control 
surfaces  to  protect  the  aircraft  from  unsafe  conditions  such  as  a  stall,"  the 
investigators  said. 

The  unit  continued  to  send  false  stall  and  speed  warnings  to  the  aircraft's 
primary  computer  and  about  2  minutes  after  the  initial  fault  "  "  generated 
very  high,  random  and  incorrect  values  for  the  aircraft's  angle  of  attack. 


> 


< 


"  "This  appears  to  be  a  unique  event,"  the  burea^eaid,  adding  that 


fitted  with  the  same  air-data  computer.  I  he  advisory  is  aimed  at 
minimizing  the  risk  in  the  unlikely  event  of  a  similar  occurrence." 


'ihe  flight  control  computer  then  commanded  a  nose-down  aircraft 

movement,  which  resulted  in  the  aircraft  pitching  down  to  a  maximum  of 
about  8.5  degrees,"  it  said. 

No  "  Similar  Event' 

Airbus  has  advised  that  it  is  not  aware  of  any  similar  event  over  the 
many  years  of  operation  of  the  Airbus,"  the  bureau  added,  saying  it  will 
continue  investigating. 
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Software  Problems 
not  just  in  Aircraft 


ConsumerReports  irg* 


May  7, 2010 

Lexus  GX  460  passes  retest;  Consumer  Reports  lifts  "Don't  Buy" 
label 


Consumer  Reports  is  lifting  the  Dont  Buy:  Safety 
Risk  designation  from  the  2010  Lexus  GX460 
SUV  after  recall  work  corrected  the  problem  it 
displayed  in  one  of  our  emergency  handling  tests. 
(See  the  original  report  and  video:  "Dont  Buy: 
Safety  Risk-2010  Lexus  GX460.") 

We  originally  experienced  the  problem  in  a  test 
that  we  use  to  evaluate  what’s  called  lift-off 
oversteer.  In  this  test,  as  the  vehicle  is  driven 
through  a  turn,  the  driver  quickly  lifts  his  foot  off 
the  accelerator  pedal  to  see  how  the  vehicle 
reacts.  When  we  did  this  with  our  GX460,  its  rear 
end  slid  out  until  the  vehicle  was  almost  sideways. 
Although  the  GX  460  has  electronic  stability 
control,  which  is  designed  to  prevent  a  vehicle 

from  sliding  thp  system masal  ialsamag niiirkK/ 


►  Otfc  101  g, 


This  article  appeared  in 

Many  appliances  now  rely  on  electronic  controls  and  operating  softw  May  2010  Consumer  Reports  Magazine.  _  But  it 

turned  out  to  be  a  problem  for  the  Kenmore  4027  front-loader,  which  scored  near  the  bottom  in  our  February  2010  report. 

Our  tests  found  that  the  rinse  cycles  on  some  models  worked  improperly,  resulting  in  an  unimpressive  cleaning. 


When  Sears,  which  sells  the  washer,  saw  our  February  2010  Ratings  (available  to  subscribers),  it  worked  with  LG,  which  makes 
the  washer,  to  figure  out  what  was  wrong.  They  quickly  determined  that  a  software  problem  was  causing  short  or  missing  rinse 
and  wash  cycles,  affecting  wash  performance.  Sears  and  LG  say  they  have  reprogrammed  the  software  on  the  models  in  their 
warehouses  and  on  about  65  percent  of  the  washers  already  sold,  including  the  ones  we  had  purchased. 

Our  retests  of  the  reprogrammed  Kenmore  4027  found  that  the  cycles  now  worked  properly,  and  the  machine  excelled.  It  now 
tops  our  Ratings  (available  to  subscribers)  of  more  than  50  front-loaders  and  we’ve  made  it  a  CR  Best  Buy. 


If  you  own  the  washer,  or  a  related  model  such  as  the  Kenmore  4044  or  Kenmore  Elite  4051  or  4219,  you  should  get  a  letter  from 
Sears  for  a  free  service  call.  Or  you  can  call  800-733-2299. 


enough  to  stop  the  slide.  We  consider  this  a  safety  risk  because  in  a  real-world  situation  this  could  cause  a  rear 
tire  to  strike  a  curb  or  slide  off  of  the  pavement,  possibly  causing  the  vehicle  to  roll  over.  Tall  vehicles  with  a  high 
center  of  gravity,  such  as  the  GX  460,  heighten  our  concern.  We  are  not  aware,  however,  of  any  reports  of  injury 
related  to  this  problem. 

Lexus  recently  duplicated  the  problem  on  its  own  test  track  and  developed  a  software  upgrade  for  the  vehicle's 
ESC  system  that  would  prevent  the  problem  from  happening.  Dealers  received  the  software  fix  last  week  and 
^egamTotifyin^GX46^wner^^rin^hei^ehicle^r^on^pai^^^^^^^^^^^^^^^^^^^^^^^^ 

We  contacted  the  Lexus  dealership  from  which  we  had  anonymously  bought  the  vehicle  and  made  an 
appointment  to  have  the  recall  work  performed.  The  work  took  about  an  hour  and  a  half. 

Following  that,  we  again  put  the  SUV  through  our  full  series  of  emergency  handling  tests.  This  time,  the  ESC 
system  intervened  earlier  and  its  rear  did  not  slide  out  in  the  lift-off  oversteer  test.  Instead,  the  vehicle 
understeered — or  plowed — when  it  exceeded  its  limits  of  traction,  which  is  a  more  common  result  and  makes  the 
vehicle  more  predictable  and  less  likely  to  roll  over.  Overall,  we  did  not  experience  any  safety  concerns  with  the 
corrected  GX460  in  our  handling  tests. 


How  do  you  upgrade  washing 
machine  software? 
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High  Fault  Leakage  Drives  Major  Increase  in  Rework  Cost 


Aircraft  industry  has  reached  limits  of  affordability 
due  to  exponential  growth  in  SW  size  and  complexity. 


Requirements 
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system  interaction  errors 
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Design 


80%  late  error 
discovery  at  high 
rework  cost 


Acceptance 

Test 


Software 
Architectural 
Design 


System 

Test 


10%,  50.5%  20x 


'or 


Major  cost  savings  through  rework  avoidance 
by  early  discovery  and  correction 

A  $10k  architecture  phase  correction  saves  $3M 


Integration 

Test 


1 


Component 

Software 

Design 


Where  faults  are  introduced 

Where  faults  are  found 

The  estimated  nominal  cost  for  fault  removal 

Sources: 

NIST  Planning  report  02-3,  The  Economic  Impacts  of  Inadequate 
Infrastructure  for  Software  Testing,  May  2002. 

D.  Galin,  Software  Quality  Assurance:  From  Theory  to 
Implementation,  Pearson/Addison-Wesley  (2004) 

B.W.  Boehm,  Software  Engineering  Economics,  Prentice  Hall  (1981) 


20%,  16% 
5x 


Unit 

Test 


Total  System  Cost 
Boeing  777  $12B 
Boeing  787  $24B 


Software  as  %  of  total  system  cost 
1997:  45%  2010:  66%  2024:  88% 


Code 

Development 


Post-unit  test  software  rework  cost 
50%  of  total  system  cost  and  growing 
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System  User/Environment 


Mismatched  Assumptions  in  System  Interactions 

System  Engineer  Physical  Plant  Control  Engineer 
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Measurement  Units,  value  range 

Impact  of 
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Boolean/Integer  abstraction 

system  failures 

Air  Canada,  Ariane,  7500  Boolean 
variable  architecture 

Under 

Control 


Operator  Error 
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Potential  event  loss 
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Engineer 


Concurrency 

Communication 

ITunes  crashes  on  dual-cores 


Embedded  software  system 
as  major  source  of  hazards 


Why  do  system  level  failures  still  occur  despite  fault 
tolerance  techniques  being  deployed  in  systems? 
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Model-based  Engineering  Pitfalls 


Inconsistency  between 
independently  developed 
analytical  models 


0  5  Id  ]5 

I  — 1 - I  1 


Confidence  that  model 
reflects  implementation 


The  system 


System  models 


System  implementation 


This  aircraft  industry  experience  has  led  to  the  System 
Architecture  Virtual  Integration  (SAVI)  initiative 
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Why  UML,  SysML  Are  Not  Sufficient 


•  System  engineering 

-  Focus  on  system  architecture  and  operational  environment 
-SysML  developed  to  capture  interactions  with  outside  world,  as  a 

standardized  UML  profile 

-4  pillars/diagrams:  requirements,  parameterics  (added  in  SysML), 
structure,  behavior 

•  Conceptual  architecture 

-  UML-based  component  model 
-Architecture  views  (DoDAF,  IEEE  1471) 

-Platform  Independent  model  (PIM) 

•  Embedded  software  system  engineering 

-OMG  Modeling  and  Analysis  of  Real  Time  Embedded  systems 
(MARTE)  as  UML  profile 

•  Borrowed  Meta  model  concepts  from  AADL 

•  Focus  on  modeling  implementations 

-xUML  insufficient  for  PSM  (Kennedy-Carter,  NATO  ALWI  study) 


Software  Engineering  Institute  Carnegie  Mellon 


Incremental  Life-Cycle  Assurance 
Feiler,  Nov  4,  2014 

©2014  Carnegie  Mellon  University 


Impact  of  Three  Step  Data  Request  Protocol 
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Operating  as  ARINC653  Partitioned  System 


Data  Consumer  Requirement 

•  Process  data  in  1  second 

Partitions 

•  Provide  space  and  time  boundary  enforcement 

•  Execute  periodically  on  a  static  timeline  at  1  second  rate 
Data  request  protocols  across  partitions 


Requestl  Providel  Request2  Provide2  Request3  Provide3  Process 


|^ConsumerJ  |^ProvideiJ  j^ConsumerJ  |^ProviderJ  |^ConsumeiJ  |^ProvideiJ  j^ConsumerJ 


I 


¥ 


How  much  time  does  consumer  actually  have  to  process  the  data? 
Who  pays  for  the  communication  overhead? 


-  Software  Engineering  Institute  Carnegie  Mellon 
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Model-based  Engineering  in  Practice 

Modeling  is  used  in  practice 

•  Modeling,  analysis,  and  simulation  in  mechanical,  control,  computer 
hardware  engineering 

Current  practice:  software  modeling  close  to  source  code 
-  Remember  software  through  pictures 
-MDE  and  MDAwith  UML 
-Automatically  generated  documents 

We  need  language  for  architecture  modeling  and  analysis 

•  Strongly  typed 

•  Well-defined  execution  and  communication  timing  semantics 

•  Systematic  approach  to  dealing  with  exceptional  conditions 

•  Support  for  large-scale  development 


-  Software  Engineering  Institute  Carnegie  Mellon 
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SAE  Architecture  Analysis  &  Design  Language 
(AADL)  for  Software-reliant  Systems 
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AADL  focuses  on  interaction  between  the  three  elements  of  a 
software-reliant  mission  and  safety-critical  systems. 
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The  SAE  AADL  Standard  Suite  (AS-5506  series) 

Core  AADL  language  standard  (V2.1-Sep  2012,  VI -Nov  2004) 

•  Strongly  typed  language  with  well-defined  execution  and  communication  semantics 

•  Textual  and  graphical  notation 

•  Standardized  XM I  interchange  format 


Standardized  AADL  Extensions 

Error  Model  language  for  safety,  reliability,  security  analysis 
ARINC653  extension  for  partitioned  architectures 
Behavior  Specification  Language  for  modes  and  interaction  behavior 
Data  Modeling  extension  for  interfacing  with  data  models  (UML,  ASN.1,  ...) 


AADL  Annex  Extensions  in  Progress 

Requirements  Definition  and  Assurance  Annex 
Synchronous  System  Specification  Annex 
Hybrid  System  Specification  Annex 
System  Constraint  Specification  Annex 
Network  Specification  Annex 
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AADL:  The  Language 

Precise  execution  semantics  for  components 

•  Thread,  process,  data,  subprogram,  system,  processor,  memory,  bus,  device, 
virtual  processor,  virtual  bus 

Continuous  control  &  event  response  processing 

•  Data  and  event  flow,  call/return,  shared  access 

•  End-to-End  flow  specifications 

Operational  modes  &  fault  tolerant  configurations 

•  Modes  &  mode  transition 

Modeling  of  large-scale  systems 

•  Component  variants,  layered  system  modeling,  packages,  abstract, 
prototype,  parameterized  templates,  arrays  of  components,  connection 
patterns 

Accommodation  of  diverse  analysis  needs 

•  Extension  mechanism,  standardized  extensions 
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Architecture-Centric  Quality  Attribute  Analysis 


Single  Annotated  Architecture  Model  Addresses 
Impact  Across  Operational  Quality  Attributes 


Safety 
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•Hazard 
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Real-time 

Performance 

•Execution  time/ 
Deadline 

•Deadlock/starvation 
•Latency 


Security 

•Intrusion 

•Integrity 

•Confidentiality 


Resource 
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•Bandwidth 

•CPU  time 

•Power 
consumption 
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Multi-Fidelity  End-to-end  Latency  in  Control 
Systems 


Operational 

Environment 


System  Engineer 


System 

Under 

Control 


Control  Engineer 


Control 

System 


Common  latency  data  from  system  engineering 

•  Processing  latency 

•  Sampling  latency 

•  Physical  signal  latency 
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Impact  of  Scheduler  Choice  on  Controller  Stability 
A.  Cervin,  Lund  U.,  CCACSD  2006 
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Software-Based  Latency  Contributors 


Execution  time  variation:  algorithm,  use  of  cache 
Processor  speed 
Resource  contention 
Preemption 

Legacy  &  shared  variable  communicatior 
Rate  group  optimization 
Protocol  specific  communication  delay 
Partitioned  architecture 
Migration  of  functionality 
Fault  tolerance  strategy 


Flow  Use  Scenario  through  Subsystem  Architecture 
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Aircraft:  (Tier  0) 


Early  Discovery  and  Incremental  V&V  through 
System  Architecture  Virtual  Integration  (SAVI) 


Aircraft  system:  (Tier  1) 

Engine,  Landing  Gear,  Cockpit, ... 
Weight,  Electrical,  Fuel,  Hydraulics,... 


/d<B«sAccess>> 
ElectricalSupply  { 

HydraulicPower  ^  I  c  (Dun/ 


LRU/IMA  System:  (Tier  2) 

Hardware  platform,  software  partitions 
Power,  MIPS,  RAM  capacity  &  budgets 
End-to-end  flow  latency 


System  &  SW  Engineering: 

Mechatronics:  Actuator  &  Wings 
Safety  Analysis  (FHA,  FMEA) 
Reliability  Analysis  (MTTF) 


J 


— ~P>  FuelSupply 


c<BusAccess>> 


i- 1  Electric  alSupply 

f — HydraulicPower 

Signals 


OEM  &  Subcontractor: 

Subsystem  proposal  validation 
Functional  integration  consistency 
Data  bus  protocol  mappings 


IE 


_ 


Subcontracted  software  subsystem:  (Tier  3) 
Tasks,  periods,  execution  time 
Software  allocation,  schedulability 
Generated  executables 


BusApeg»»p>  analogl _ analog2^j*B»wApcess> >  I  I  li= 


►  navSensorDataFromNSP 
navDataT  oGPAPC 


Repeated  Virtual  Integration  Analyses: 
Power/weight 
MIPS/RAM,  Scheduling 
End-to-end  latency 
Network  bandwidth 


Proof  of  Concept  Demonstration  and  Transition  by  Aerospace  industry  initiative 

•  Architecture-centric  model-based  software  and  system  engineering 

•  Architecture-centric  model-based  acquisition  and  development  process 

•  Multi  notation,  multi  team  model  repository  &  standardized  model  interchange 

■  Multi-tier  system  &  software  architecture  (in  AADL) 

■  Incremental  end-to-end  validation  of  system  properties 


Software  Engineering  Institute  Carnegie  Mellon 
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Multi-Notation  Approach  to  Architecture-centric 
Virtual  System  and  Software  Integration 
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Architecture-centric  Virtual  Integration  Practice 
(AC  VIP) 


Iterative  architecture 
design,  safety  analysis,  and 
requirement  decomposition 


Stakeholder  and 
Quality  Attribute  (QA) 
driven  architecture¬ 
centric  requirement 
specification 


Transformation  and 
code  generation  based 
on  verified  architecture 
specifications 


Architecture-centric  virtual 
integration  and  compositional 
verification  of  requirements 


Testing  against 
verified  specifications 
and  models 


Assurance  plan 
and  execution 
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Outline 


Challenges  in  Safety-critical  Software-intensive  systems 

An  Architecture-centric  Virtual  Integration  Strategy  with  SAE  AADL 

Improving  the  Quality  of  Requirements 

Architecture  Fault  Modeling  and  Safety 

Incremental  Life-cycle  Assurance  of  Systems 

Summary  and  Conclusion 


-  Software  Engineering  Institute  Carnegie  Mellon 
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Certification  &  Recertification  Challenges 

Certification:  assure  the  quality  of  the  delivered  system 

•  Sufficient  evidence  that  a  system  implementation  meets  system  requirements 

•  Quality  of  requirements  and  quality  of  evidence  determines  quality  of  system 
Certification  related  rework  cost 

•  Currently  50%  of  total  system  cost  and  growing 
Recertification  Challenge 

•  Desired  cost  of  recertification  in  proportion  to  change 

Improve  quality  of  requirements  and  evidence 

Perform  verification  compositionally 
throughout  the  life  cycle 
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Industry  Practice  in  DO-178B  Compliant 
Requirements  Capture 


Notation 

Enter  an  “s'  in  every  row/column  cell  that  applies 

Sy  stem  Rcqu j  r cm  ent  s 

Data  Interconnect  {ICO} 

O 

c 

§ 

s- 

‘5 

rr 

0 

OC 

L. 

5fl 

| 

KJ 

u 

■ 

5 

5 

g 

p 

"3 

p 

0 

S 

0 

13 

> 

p 

-J 

[ 

£ 

c 

J 

H  ar  d war c  Rcq  u  i  r cm  e nts 

English  Test  or  Shall  Statements 

39 

27 

36 

32 

29' 

Tables  and  Diagrams 

31 

30 

30 

19 

18 

UML  Use  Cases 

1 

2 

4 

UML  Sequence  Diagrams 

3 

6 

UML  State  Diagrams 

1 

7 

Executable  Models  (e.g,  Siniulink.  SCADE  Suite, 

7 

1 

y 

E 

1 

etc.) 

i 

0 

1 

Data  Flow  Diagrams  (e.g,  Yourdon) 

4 

6 

9 

Need  analyzable  &  executable  specifications 

Other  (Specify)XML 

1 

Operational  models  or  prototypes 

1 

1 

1 

UML 

i 

1 

Tool 

/ 

Enter  an  “x”  m  every  row/column  cell  that 
applies 

System  Requirements 

a 

u 

y 

i 

c 

0 

Q 

High-Level  Software  Requirements 

Low-Level  Software  Requirements 

Hardware  Requirements 

DOORS 

23 

13 

22 

18 

12 

Rational  ROSE* 

1 

3 

RDD-100* 

Requisite  Pro11 

5 

3 

5 

4 

4 

Rhapsody 

1 

SCADE  Suite 

2 

3 

1 

Siniulink 

5 

1 

5 

3 

1 

Slate 

1 

1 

1 

Spreadsheet  (e.g..  Microsoft  Excel) 

5 

4 

5 

4 

3 

Statemate 

Word  Processor  (e.g.,  Microsoft  Word) 

19 

20 

18 

17 

16 

YAPS™ 

1 

3 

3 

Designer's  Workbench™ 

1 

1 

Proprietary  Database.  SCADE  like  pic  tool 

1 

1 

Interleaf 

1 

1 

1 

1 

1 

BEACON 

1 

1 

1 

1 

CaliberRM 

1 

1 

1 

1 

1 

XM: 

1 

Wiring  diagram 

1 

1 
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Requirement  Quality  Challenge 


Requirements 

error 

% 

Incomplete 

21% 

Missing 

33% 

Incorrect 

24% 

Ambiguous 

6% 

Inconsistent 

5% 

r  1 

There  is  more  to  requirements  quality  than  “shall”s  and  stakeholder  traceability 
IEEE  830-1998  Recommended  Practice  for  SW  Requirements  Specification 

s 


UserReqts  Technical  Reqts  Design  Test  Cases 


Browsable  links/Coverage  metrics 


IEEE  Std  830-1998  characteristics  of  a  good  requirements  specification: 

•  Correct 

•  Unambiguous 

•  Complete 

•  Consistent 

i  Ranked  for  importance  and/or  stability 

•  Verifiable 

•  Modifiable 

•  Traceable 


System  to  SW  requirements  gap  [Boehm  2006] 

How  do  we  verify  low  level  SW  requirements 
against  system  requirements? 


When  StartUpComplete  is  TRUE  in  both  FADECs  and 
SlowStartupComplete  is  FALSE, 

the  FADECStartupSW  shall  set  SlowStartupInComplete 
to  TRUE 
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Stakeholder  Needs  and  Requirement  Categories 


ISO/IEC/IEEE.  2011.  Systems  and  Software  Engineering  -  Requirements  Engineering.  Geneva,  Switzerland:  International  Organization  for  Standardization  [ISO ’(/International  Electrotechnical  Commission/ 
Institute  of  Electrical  and  Electronics  Engineers  (IEEE),  [EC),  ISO/IEC/IEEE  291 4fl. 


Typ-e  of 
Stakeholder 
Requirement 


Table  2.  Example  of  Stakeholder  Requirements  Classification.  [SEBdK  Original} 


Table  2.  Example  of  System  Requirements  Classification.  (SEBdK  Original) 


Types  of  System 

Description 

Requirement 

Service  or 

Functional 

Sets  of  actio 

Functional 

Requirements 

Describe  qualitatively  the  sys 

Operational 

This  categor 

*  Operatio 

*  Operatio 

*  Operatic 

of-intere 

Performance 

Requirements 

D  e  fin  e  q  u  a  ntitativ  e  ly  th  e  exte 
system  performance  and  are 

or  task. 

Usability 

Requirements 

Define  the  quality  of  system  l 

Interface 

Requirements 

Define  how  the  system  is  rec 
system,  including  human  elen 
systems  or  internal  system  e 

Interface 

Matter,  enerc 

Environmental 

External  con 

Operational 

Requirements 

Define  the  operational  conditi 
maintainability,  reliability,  and  i 

Utilization 

Characteristics 

Human  Factors 

The  r-j|ftiesr  u 

Capabilities  t 

Modes  and/or  States 

Requirements 

D  e  fin  e  th  e  various  ope  ratio  n  e 

Logistical 

Acquisition,  1 

Adaptability 

Requirements 

Physical  Constraints 

Define  potential  extension,  gr 

Define  constraints  on  weight, 

Design  and 

Realization 

Constraints 

Reuse  of  ex 

Design  Constraints 

Define  the  limits  on  the  option 
provided  system  element,  or 

Process 

Constraints 

These  are  si 

system,  but 
laws,  ad  mini 
corporate  pc 
agreement  d 

Environmental 

Conditions 

Define  the  environmental  con 

temperature,  fauna,  salt,  dus 
societal  environment  (e.g.  leg 

Logistical 

Requirements 

Define  the  logistical  condition, 
personnel,  spare  parts,  traini 

Project 

Constraints 

Constraints  1 

Policies  and 

Regulations 

Define  relevant  and  applicahli 
regulatory  agony,  health  orsE 

Business  Model 

Constraints 

Constraints  i 

(local,  nation 

Costand  Schedule 

Constraints 

Define,  for  example,  the  cost 

i  functions  or  tasks  to  be  nerfprmed  in  one  ration. 


SYSTEM  DEVELOPMENT 


SYSTEM  OPERATIONS 


Congress  and  Legislatures 

IDavan-iweffl  nepooa 
Lcbtrfing 

Hearing  art  open  meelrtB 
Aoddtants 


Ceng^ess  and  Legislature* 

Lcqclnlinn  I  I  Lcbbymg 

Heai » us  and  cs»ii  rneaiinfle. 
Accidents 


Government  ReguKalory  agencies 
Industry  Associations., 

User  Associations.  Unions, 
Insurance  Companies,  Courts 


Government  Regulatory  Agencies 
Industry  Associations, 

User  Associations,  Unions, 
insurance  Companies,  Courts 


RogulaLrarc 
$r.jrt3rd& 
Ge'bliaalkin 
Lsgal  fwrkflfH* 


Caw  Law 


Leveson  System 

j  ADCidcnts  3nd i  re  id  err.  j 


Theoretic  Framework 


Company 

Management 


Acddflmc  and  inddanr  rtf  on;:: 
Cparartcns  reparts 
Ugmltriftnw  Repent 
Change  rapcrls 


Safety  Pa  icy 
SiandaoJa 
Hkcotc; 


&alu5  Rcacrls 
nisJi 

hz'denl  Rapcrls 


Company 

Management 


SalnLy  Pdicy 
Slmdand* 


Otwrallona  Repent 


&a1?ly  Sljndiirids 


Hasaid  Analyses 
Safety- Re  aria !i  Ghangas 
Progress  Reports 


Des  ign, 
□ocumematlon 


Mixture  of  Requirements  &  Architecture  Design  Constraints 

Requirements  and  Design  Information 


Requirements  for  a 
Patient  Therapy  System 

The  patient  shall  never  be  infused 
with  a  single  air  bubble  more  than 
Sml  volume. 

When  a  single  air  bubble  more 
than  Sml  volume  is  detected, 
the  system  shall  stop  infusion 
within  0.2  seconds. 

When  piston  stop  is  received, the 
system  shall  stop  piston  movement 
within  0.0 1  seconds. 

The  system  shall  always 
stop  the  piston  at  the 
bottom  or  top  of  the 
chamber 


The  patient  shall  never  be  infused 
with  a  single  air  bubble  more  than 
Sml  volume. 


2. 


AT  I E  NT  T  H  E  R ARY  SY5T  EM 


INFUSION  SYSTEM 


DRUG 

DELIVERY 

HARDWARE 

AIR  BUBBLE  1 
SENSOR 

PUMP  SYSTEM 

PUMP 

PUMP 

HARDWARE 

CONTROLLER 

When  a  singleair  bubble  more 
than  Sml  volume  is  detected, 
the  system  shall  stop  infusion 
within  0.2  seconds. 


The  system  shall  always 
stop  the  piston  at  the 
bottom  or  top  of  the 
chamber. 


When  piston  stop  is  received, the 
system  shall  stop  piston  movement 
within  0.0  II  seconds. 


Typical  requirement  documents  span  multiple  levels  of 

a  system  architecture 

We  have  made  architecture  design  decisions. 

We  have  effectively  specified  a  partial  architecture 


Adapted  from  M.  Whalen  presentation 
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System  Specification  and  Requirements 
Coverage 


{  Developmental'* 
Requirements 


Quality  attribute  utility  tree 


Modifiability  i 


Assurability  i 


Environmental  Assumptions 


■ 

■ 

■ 

■ 

■ 

■ 

■/ 

■ 

■ 

QA  — 

■ 

■ 

■ 

Utility 

■ 

■ 

/ 

Requirements 

Guarantees 

Assumptions 


Environment 


Precondition 

Postcondition 

Invariant 


Interaction  contract: 
match  input  assumption 
with  guarantee 


—  Performance 


' —  Modifiability 


i 

l 


Data 

Latency 


Transaction 
Throughput 
New  products 


(M.M) 

(H.H) 


Deliver  video  in  real  time 


Change 

COTS 


—  Availability 


—  Security 


t 

-c 


COTSSAV 

failures 

Data 

confidentiality 
Data 
integrity 


lucts  I -  Maa 

in  <21 

li”±l  $Ta 

(H.H 

-c 

(H.H 

«T"| 


Add  CORBA  middleware 
20  person-months. 

Change  Web  user  interface 
'  person-weeks 

Power  outage  at  site  1  requires  traffic 
redirected  to  site2  in  <  3  seconds. 

Network  failure  detected  and  recovered 


JH.L) 


Credit  card  transactions  are  secure 
99  909%  ofthe  time 

Customer  D8  authorization  works 
99  999%  ofthe  time 


(  Mission  N 
Requirements  ■ 

. . 1  I 


(  Dependability N 


.  Requirements 

I  r - i 


Function  i 


I 


I 


Behavior  i 

_  _  _  _  _ 

- 1  |  ^ - 1 

Performancei  I  Security  i 


Reliability  i 


r - \ 

Safety  i 


_ /  ' _ * 


Implementation  constraints 


Exceptional  condition 
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Architecture-led  Requirement  &  Hazard  Specification 


System  Specification  Coverage 


Mission-  \ 
critical 

Requiremen  | 
Function 

Behavior 

Performance 

_  .”1 .  / 


Outline 


Challenges  in  Safety-critical  Software-intensive  systems 

An  Architecture-centric  Virtual  Integration  Strategy  with  SAE  AADL 

Improving  the  Quality  of  Requirements 

Architecture  Fault  Modeling  and  Safety 

Incremental  Life-cycle  Assurance  of  Systems 

Summary  and  Conclusion 
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AADL  Error  Model  Scope  and  Purpose 


System  safety  process  uses  many  individual  methods  and  analyses,  e.g. 

•  hazard  analysis 

•  failure  modes  and  effects  analysis 

•  fault  trees  Capture  risk  mitigation  architecture 

•  Markov  processes 


C^stenT^>  Capture  hazards 


(Component)  Capture  FMEA  model 


Goal:  a  general  facility  for  modeling  fault/error/failure  behaviors  that  can  be 
used  for  several  modeling  and  analysis  activities. 

Annotated  architecture  model  permits  checking  for  consistency 
and  completeness  between  these  various  declarations. 


Related  analyses  are  also  useful  for  other  purposes,  e.g. 


•  maintainability 

•  availability 

•  Integrity 

•  Security 


SAE  ARP  4761  Guidelines  and  Methods  for  Conducting  the  Safety 
Assessment  Process  on  Civil  Airborne  Systems  and  Equipment 
Demonstrated  in  SAVI  Wheel  Braking  System  Example 


Error  Model  Annex  can  be  adapted  to  other  ADLs 
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Error  Propagation  Contracts 


Incoming 


NoData 


Component  C 

ValueError 

NoData 

i  / 

BadValue 

Er^ 

NoResource 


Binding 


BadValue 
NoData 


Outgoing 


LateData 


Legend 


► 


Port 


HW  Binding 


Propagation 
of  Error  Types 
Direction 


Propagated 
Error  Type 


Not 

propagated 


Error  Flow  through  component 
Path  PI. NoData->P2. NoData 
Source  P2.BadData 

Path  processor. NoResource  ->  P2. NoData 


“Not"  on  propagated  indicates  that  this 
error  type  is  intended  to  be  contained. 

This  allows  us  to  determine  whether 
propagation  specification  is  complete. 


Incoming/Assumed 

•  Error  Propagation 

Propagated  errors 

•  Error  Containment: 

Errors  not  propagated 


Outgoing/Contract 

•  Error  Propagation 

•  Error  Containment 


Bound  resources 

•  Error  Propagation 

•  Error  Containment 

•  Propagation  to  resource 
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Original  Preliminary  System  Safety  Analysis 
(PSSA) 


( - ; - - - \ 

System  engineering  activity  with 

focus  on  failing  components. 
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Discovery  of  Unexpected  PSSA  Hazard  through 
Repeated  Virtual  Integration 


system  Ell 
features 

trueairspeed :  out  data  port  DataDictionary :: Velocity; 

flows 

fl:  flow  s 
Latency 
}; 

annex  EMV2  {* 
error  prc 
use  type: 
use  beha\ 
truei 
flows 
efl:errc 
ef2:errc 
properties 
EMV2 : : hazard 
[  crossref€ 
failure  : 
phase  => 
descriptd 
severity 
critical] 
comment  : 


Anticipated: 
No  EGI  data 

V' 


Flight  Mgnt  Syste 


Jcorrupted^ 


system  implem 
subcomponen 

PilotGrip 
Positions 
EGI:  systl  ^ 
FMS:  proces^^ 


Unexpected  propagation  of 
corrupted  Airspeed  data  results 
in  Stall  due  to  miss-correction 


Vibration  causes  boards  to 
touch  which  causes  EGI 
data  corruption 


Actuatorl:  device  Actuator  ; 

Actuator2:  device  Actuator  ; 

FMSProcessor :  processor  PowerPt 
connections 

pilotCmd:  port  PilotGrip. Desin  _ 
sensedPosition :  port  PositionSensor . PositionReading  ->  FMS . Position; 
ActuatorlCmd :  port  FMS.ActCmd  ->  Actuatorl .ActCmd; 

Actuator2Cmd :  port  FMS.ActCmd  ->  Actuator2 .ActCmd; 

vtx :  port  EGI . TrueAirSpeed  - >  FMS. TrueAirSpeed ; _ 

f 


©Outgoing  propagation  {Failure,  CorruptedData}  is  not  handled.  Expected  incoming  {Failure} 


FMS 

Processor 
Operational 
J  Failed 


FMS  Power 


Anticipated: 

NoService 


ctuator 

Cmd 


NoService  | 

””  Stair~”j 


Anticipated:  No 
Stall  Propagation  | 


EGI  maintainer  adds  corrupted  data  hazard  to  model. 
Error  Model  analysis  of  integrated  model  detects 
unhandled  propagation. 


-  s  m.cudiununu - ✓  m.  tuffiui'i .  rs 


Latency  =>  15  ms 

}; 


20  ms; 
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Recent  Automated  FMEA  Experience 

Failure  Modes  and  Effects  Analyses  are  rigorous  and  comprehensive 
reliability  and  safety  design  evaluations 

•  Required  by  industry  standards  and  Government  policies 

•  When  performed  manually  are  usually  done  once  due  to  cost  and  schedule 

•  If  automated  allows  for 


-  multiple  iterations  from  conceptual  to  detailed  design 

-  Tradeoff  studies  and  evaluation  of  alternatives 

-  Early  identification  of  potential  problems 


*■ - 

ID  Item  Initial  State  Initial  Failure  Mode  1st  Level  Effect  Transition  2nd  Level  Effect:  Transition  3rd  Level  Effect  Severity  M 

1 

Sat Bus 

Working 

Failure 

Failed 

Failed 

Recovery 

Working 

Workir 

1 

Sat Payload 

Working 

Working 

Bus  failure  causes  payload  transition 

Standby 

Standby 

Bus  Recovery  Causes  Payload  Transition 

Workir 

2 

Sat Bus 

Working 

Working 

Working 

5 

2 

Sat_Payload 

Working 

Failure 

Failed 

Recovery 

Working 

5 

Largest  analysis  of  satellite  to  date  consists  of  26,000  failure  modes 

•  Includes  detailed  model  of  satellite  bus 

•  20  states  perform  failure  mode 

•  Longest  failure  mode  sequences  have  25  transitions  (i.e.,  25  effects) 

Myron  Hecht,  Aerospace  Corp. 

Safety  Analysis  for  JPL,  member  of  DO-1 78C  committee 
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Challenges  in  Safety-critical  Software-intensive  systems 

An  Architecture-centric  Virtual  Integration  Strategy  with  SAE  AADL 

Improving  the  Quality  of  Requirements 

Architecture  Fault  Modeling  and  Safety 

Incremental  Life-cycle  Assurance  of  Systems 

Summary  and  Conclusion 
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Reliability  &  Qualification  Improvement  Strategy 


2010  SEI  Study  for  AMRDEC 
Aviation  Engineering  Directorate 
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Verification  Actions 


Table  2.  Main  Ontology  Elements  as  Handled  within  Verification.  (SEBoK  Original) 


Element 

Definition 

Attributes  (examples) 

Verification  Action 

A  verification  action  describes  what  must  be  verified  (the  element  as  reference),  on  which  element,  the  expected  result,  the  verification 
technique  to  apply,  on  which  level  of  decomposition. 

Identifier,  name,  description 

Verification 

A  v( 

Table  3.  Verification  Techniques.  (SEBoK  Original) 

Procedure 

Ider 

Verification 

Technique 

Description 

Verification  Tool 

A  v< 

Ider 

Inspection 

Technique  based  on  visual  or  dimensional  examination  of  an  element;  the  verification  relies  on  the  human  senses  or  uses  simple  methods  of 
measurement  and  handling.  Inspection  is  generally  non-destructive,  and  typically  includes  the  use  of  sight,  hearing,  smell,  touch,  and  taste, 
simple  physical  manipulation,  mechanical  and  electrical  gauging,  and  measurement.  No  stimuli  (tests)  are  necessary.  The  technique  is  used 
to  check  properties  or  characteristics  best  determined  by  observation  (e  g.  -  paint  color,  weight,  documentation,  listing  of  code,  etc  ). 

Verification 

A  v< 

Configuration 

pro< 

Ider 

Analysis 

Technique  based  on  analytical  evidence  obtained  without  any  intervention  on  the  submitted  element  using  mathematical  or  probabilistic 
calculation,  logical  reasoning  (including  the  theory  of  predicates),  modeling  and/or  simulation  under  defined  conditions  to  show  theoretical 
compliance.  Mainly  used  where  testing  to  realistic  conditions  cannot  be  achieved  or  is  not  cost-effective. 

Risk 

Am 

(US( 

Analogy  or 
Similarity 

Technique  based  on  evidence  of  similar  elements  to  the  submitted  element  or  on  experience  feedback.  It  is  absolutely  necessary  to  show  by 
prediction  that  the  context  is  invariant  that  the  outcomes  are  transposable  (models,  investigations,  experience  feedback,  etc.).  Similarity  can 

Rationale 

An ; 

only  be  used  if  the  submitted  element  is  similar  in  design,  manufacture,  and  use;  equivalent  or  more  stringent  verification  actions  were  used 
for  the  similar  element,  and  the  intended  operational  environment  is  identical  to  or  less  rigorous  than  the  similar  element. 

Ider 

Demonstration 

Technique  used  to  demonstrate  correct  operation  of  the  submitted  element  against  operational  and  observable  characteristics  without  using 

physical  measurements  (no  or  minimal  instrumentation  or  test  equipment).  Demonstration  is  sometimes  called  'field  testing'.  It  generally 
consists  of  a  set  of  tests  selected  by  the  supplier  to  show  that  the  element  response  to  stimuli  is  suitable  or  to  show  that  operators  can 
perform  their  assigned  tasks  when  using  the  element.  Observations  are  made  and  compared  with  predetermined/expected  responses. 
Demonstration  may  be  appropriate  when  requirements  or  specification  are  given  in  statistical  terms  (e  g.  meant  time  to  repair,  average  power 
consumption,  etc  ). 

Test 

Technique  performed  onto  the  submitted  element  by  which  functional,  measurable  characteristics,  operability,  supportability,  or  performance 
capability  is  quantitatively  verified  when  subjected  to  controlled  conditions  that  are  real  or  simulated.  Testing  often  uses  special  test 
equipment  or  instrumentation  to  obtain  accurate  quantitative  data  to  be  analyzed. 

Sampling 

Technique  based  on  verification  of  characteristics  using  samples.  The  number,  tolerance,  and  other  characteristics  must  be  specified  to  be  in 
agreement  with  the  experience  feedback. 
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Integrated  Approach  to  Requirement  V&V 
through  Assurance  Automation 
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Contract-based  Compositional  Verification 

Secure  Mathematically-Assured  Composition  of  Control  Models 


Key  Problem 

Many  vulnerabilities  occur  at  component  interfaces. 
How  can  we  use  formal  methods  to  detect  these 
vulnerabilities  and  build  provably  secure  systems? 


TA4  -  Research  Integration  and 
Formal  Methods  Workbench 
Rockwell  Collins  and 
University  of  Minnesota 
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16  months  into  the  project 

Draper  Labs  could  not  hack  into  the 
system  in  6  weeks 

Had  access  to  source  code 


Technical  Approach 

Develop  a  complete,  formal  architecture  model  for  UAVs  that 
provides  robustness  against  cyber  attack 

Develop  compositional  verification  tools  driven  from  the 
architecture  model  for  combining  formal  evidence  from  multiple 
sources,  components,  and  subsystems 

Develop  synthesis  tools  to  generate  flight  software  for  UAVs 
directly  from  the  architecture  model,  verified  components,  and 
verified  operation  system 


Accomplishments 

•  Created  AADL  model  of  vehicle  hardware  &  software 
architecture 

•  Identified  system-level  requirements  to  be  verified 
based  on  input  from  Red  Team  evaluations 

•  Developed  Resolute  analysis  tool  for  capturing  and 
evaluating  assurance  case  arguments  linked  to 
AADL  model 

•  Developed  example  assurance  cases  for  two 
security  requirements 

•  Developed  synthesis  tool  for  auto-generation  of 
configuration  data  and  glue  code  for  OS  and  platform 
hardware 
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Building  the  Assurance  Case  throughout  the  Life  Cycle 


Continuous  Confidence  Measure  throughout  Life 
Cycle  that  a  System  Meets  its  Requirements 
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Architecture-centric  Virtual  System  Integration 
&  Incremental  Life-cycle  Assurance 


Reduce  risks 

•  Analyze  system  early  and  throughout  life  cycle 

•  Understand  system  wide  impact 

•  Validate  assumptions  across  system 

Increase  confidence 

•  Validate  models  to  complement  integration  testing 

•  Validate  model  assumptions  in  operational  system 

•  Evolve  system  models  in  increasing  fidelity 

Reduce  cost 

•  Fewer  system  integration  problems 

•  Incremental  evidence  through  compositional  verification 

•  Fewer  verification  steps  through  generation  from  single  source  and  verified 
models 


-  Software  Engineering  Institute  Carnegie  Mellon 


Incremental  Life-Cycle  Assurance 
Feiler,  Nov  4,  2014 

©2014  Carnegie  Mellon  University 


Model-Based 
Engineering 
with  AADL 


References 

AADL  Website  www.aadl.info  and  AADL  Wiki  www.aadl.info/wiki 

Blog  entries  and  podcasts  on  AADL  at  www.sei.cmu.edu 

AADL  Book  in  SEI  Series  of  Addison-Wesley 
http://www.informit.com/store/product.aspx?isbn=0321 888944 

On  AADL  and  Model-based  Engineering 

http://www.sei.cmu.edu/librarv/assets/ResearchandTechnology  AADLandMBE.p 

df 


L1.lv  id  1'  (Jlu-ch 


On  an  architecture-centric  virtual  integration  practice  and  SAVI 

http://www.sei.cmu.edu/architecture/research/model-based- 

enqineering/virtual  system  inteqration.cfm 

On  an  a  four  pillar  improvement  strategy  for  software  system  verification  and 
qualification 

http://bloq.sei.cmu.edu/post.cfm/improvinq-safetv-critical-svstems-with-a- 

reliabilitv-validation-improvement-framework 

Webinars  on  system  verification  https://www.csiac.org/event/architecture-centric- 
virtual-inteqration-strateqy-safetv-critical-svstem-verification  and  on  architecture 
trade  studies  with  AADL  https://www.webcaster4.com/Webcast/Page/139/5357 


Software  Engineering  Institute  Carnegie  Mellon 


Incremental  Life-Cycle  Assurance 
Feiler,  Nov  4,  2014 

©2014  Carnegie  Mellon  University 


45 


Contact  Information 


Peter  H.  Feiler 

Principal  Researcher 
RTSS 

Telephone:  +1  412-268-7790 
Email:  phf@sei.cmu.edu 

Web 

Wiki.sei.cmu.edu/aadl 

www.aadl.info 


U.S.  Mail 

Software  Engineering  Institute 
Customer  Relations 
4500  Fifth  Avenue 
Pittsburgh,  PA  15213-2612 
USA 

Customer  Relations 

Email:  info@sei.cmu.edu 

SEI  Phone:  +1  412-268-5800 

SEI  Fax:  +1  412-268-6257 


Software  Engineering  Institute 


Carnegie  Mellon 


Incremental  Life-Cycle  Assurance 
Feiler,  Nov  4,  2014 

©2014  Carnegie  Mellon  University 


